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VI.  HONORS  AND  AWARDS 


1983:  Anil  K.  Jain  was  awarded  the  IEEE  Donald  G.  Fink  Prize  for  his  paper  on  Image 

Data  Compression  in  Proceedings  of  the  IEEE,  March  1981. 

1986:  Anil  K.  Jain  and  James  Apffel  were  recipients  of  NASA  Group  Achievement 

Award  for  their  contributions  in  reconstruction  of  the  Comet  Halley  image  (based 
on  research  performed  under  ARO-sponsorship).  This  image  was  released  by 
NASA  and  published  world-wide  by  the  news  media. 

1987:  Anil  K.  Jain  was  elected  Fellow  of  the  IEEE  for  "contributions  to  image 

processing."  Much  of  these  contributions  were  made  possible  by  research 
supported  by  ARO. 
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VII.  SUMMARY  OF  RESEARCH  FINDINGS 


During  the  entire  period,  research  was  conducted  on  several  different  topics  including  image 
modeling  and  representation,  image  restoration,  2-D  spectral  estimation  and  spectral  factorization, 
image  data  compression,  architectures  of  image  processing  algorithms,  computer  vision.  Radon 
transform  theory  and  applications,  mathematical  morphology  for  image  processing  and  o  on. 
These  projects  are  briefly  summarized  here. 

A.  TWO  DIMENSIONAL  STOCHASTIC  MODELS  FOR  IMAGE  PROCESSING 
(JAIN  AND  RANGANATH) 

Three  canonical  representation,  causal,  semicausal,  and  noncausal  were  introduced  for  two- 
dimensional  images.  A  new  realization  theory  in  the  framework  of  2-D  prediction  and 
spectral  factorization  was  developed  where  it  was  shown  that  a  well  behaved  2-D  power 
spectrum  S(coi,  0)2)  can  be  factored  as 

S((oi,(jo.)  «  KH(a)i,  0)2)  H(-a)i, -0)2) 

A 

s  S(0)1,  0)2) 

such  that  H  is  the  frequency  response  of  a  finite  order,  stable,  minimum  variance  causal, 

A 

semicausal,  or  noncausal  system  and  the  spectral  match  between  S  and  S  can  be  arbitrarily 
close.  Moreover,  the  algorithm  for  realizing  H  requires  that  only  a  finite  number  of  linear 
equations  be  solved  unlike  prior  methods  of  two  dimensional  spectral  factorization  which 
require  solution  of  infinite  number  of  equations  to  obtain  finites  order  approximate 
realizations. 

One  practical  significance  of  these  models  is  that  they  spell  different  architectures  for 
processing  of  2-D  data.  For  example,  DPCM,  Hybrid  coding  and  Transform  coding 
methods  for  image  transmission  arise  from  the  causal,  semicausal  and  noncausal  models, 
respectively.  Other  important  applications  in  image  restoration,  edge  extraction,  spectral 
estimation  and  texture  modeling  were  reported.  See  Publications  [  1,  2,  7,  9,  29,  30,  32]  and 
PhD  thesis  by  S.  Ranganath. 
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B.  RADON  TRANSFORM  THEORY  FOR  RANDOM  FIELDS  (JAIN  AND 
ANSARI) 

The  Radon  Transform  gives  the  mathematical  formulation  of  the  projections  of  a  two  (or 
higher)  dimensional  function.  Simply  stated,  the  Radon  Transform  g(s,  0)  of  a  function 
f(x,  y)  is  its  line  integral  along  a  line  placed  at  a  distance  s  from  the  origin  with  an  orientation 
0,  i.e.. 


g(s,  0)  =  j|  f(x,y)  5(s-  X  cos  0-  y  sin  0)  dx  dy 

The  mathematical  theory  associated  with  this  transformation  holds  the  key  for  computerized 
tomography  (CT)  which  gives  cross  sectional  images  of  a  3-D  object  from  its  2-D 
projections. 

A  fundamental  result  associated  with  this  theory  is  the  Projection-Slice  Theorem  which  states 
that  the  1-D  Fourier  Transform,  G(^,  9)  of  the  projection  g(s,  0)  is  equal  to  the  central  slice 
of  the  2-D  Fourier  Transform  F(^  cos  0,  ^  sin  0)  at  angle  0,  i.e., 

G(^,0)  =  F(^  cos  0,  4  sin  0) 

where  (^,  0)  are  the  polar  coordinates  in  the  Fourier  space.  This  result  can  be  used  to  reduce 
the  complexity  of  several  2-D  and  3-D  image  processing  algorithms.  In  our  research,  we 
introduced  a  Unitary  Radon  Transfomi  [6,  29,  31,  32J,  which  was  shown  to  be  the  function 
g(s,  0)  obtained  by  filtering  g(s,  9)  by  a  linear  filter  whose  frequency  response  is  Via 

the  use  of  the  unitary  Radon  Transform,  it  was  shown  that  for  a  stationary  random  field 
f(x,  y),  its  projections  g(s,  9)  and  filtered  projections  g(s,  0)  both  are  uncorrelated  in  0  and 
stationary  in  s.  Moreover,  the  1-D  power  spectrum  density  of  g(s,  0)  denoted  by  S|(^,  0)  is 
the  central  slice  of  the  2-D  power  spectrum  of  f(x,  y)  denoted  SKq  cos  0,  sin  0). 

S_(^  9)  =  Sj.(^  cos  0,  ^  sin  0) 


The  result  above  is  noteworthy  because,  contrary  to  earlier  assumptions  by  various 
researchers,  the  power  spectrum  satisfies  the  projection  theorem  in  the  unitary  Radon 
Transform  domain  whereas  the  amplitude  spectrum  satisfies  it  in  the  conventional  Radon 
Transform  domain.  This  result  has  a  fundamental  impact  when  processing  is  done  in  the 
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Radon  space,  as  was  demonstrated  in  [6]  for  reconstruction  of  CT  images  in  the  presence  of 
detector  noise.  Thus,  the  Unitary  Radon  space  is  the  correct  projection  space  in  which  to 
apply  stochastic  estimation  theory  for  derivation  of  new  convolution  kernels  for  2-D  image 
reconstruction  from  projections,  2-D  panern  matching,  and  methods  for  estimating  the  power 
spectrum  of  2-D  objects  from  their  unitary  projections. 

C.  RECURSIVE  BLOCK  CODING  -  AN  IMAGE  DATA  COMPRESSION 
METHODOLOGY  (JAIN  AND  FARRELLE) 

The  limiting  factor  for  image  data  compression  using  transform  coding  or  VQ  is  the  so-called 
block  or  tile  effect.  At  sufficiently  high  rates,  the  loss  of  fidelity  is  a  continuous  function  of 
the  data  rate.  However,  there  comes  a  point  when  any  further  reduction  in  the  rate  produces 
the  tile  effect  and  this  leads  to  a  high  discontinuity  in  the  perceived  fidelity  and  subjective 
results  show  that  this  artifact  is  ten  times  more  objectionable  than  equal  energy  random  noise. 
The  cause  of  the  problem  is  that  each  block  of  the  image  is  coded  independently,  that  is 
without  reference  to  the  neighboring  blocks,  and  so  the  distortion  introduced  by  the  coding  is 
discontinuous  from  block  to  block.  The  eye  picks  up  the  change  in  luminance  at  the  edges  of 
the  blocks,  especially  in  the  smooth  regions  where  no  such  change  is  expected. 
Furthermore,  the  effect  is  amplified  since  the  regular  block  structure  ensures  that  these 
problems  areas  are  all  aligned  and  the  resulting  image  resembles  a  mosaic  constructed  from 
small  tiles.  In  other  words,  we  are  sensitive  not  only  to  the  amplitude  of  errors  but  also  to 
their  structure. 

We  have  alleviated  this  problem  when  block  coding  an  image  by  using  a  non-causal 
representation  for  the  image  which  allows  us  to  decompose  it  as  the  sum  of  two  sources:  a 
boundary  response  and  a  resulting  residual  image. 

The  reason  why  this  two  source  decomposition  reduces  the  tile  effect  is  because  if  we  share 
the  boundary  values,  which  are  typically  one  pixel  wide,  between  two  neighboring  blocks, 
then  the  boundary  response  which  interpolates  the  boundary  values  will  be  continuous  across 
the  transition  from  one  block  to  the  next  despite  any  coding  distortion  in  the  boundaries.  See 
publications  [2,  9,  18,  28,  29,  32]  and  the  PhD  thesis  by  Farrelle. 
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D.  CONTOUR  POINT  ALGORITHMS  FOR  CONTOUR  PROCESSING  AND 
SHAPE  ANALYSIS  (JAIN  AND  PAGLIERONI) 

Planar  non-overlapping  shape  analysis  is  an  important  computer  vision  problem  with 
applications  in  industrial  parts  inspection,  character  recognition,  target  identification, 
biological  cell  analysis,  etc.  Subtopics  emphasized  in  this  research  include  shape 
representation,  measurement  and  classification.  A  unified  approach  composed  of  contour 
based  algorithms  for  solving  problems  of  each  type  is  developed,  where  contour  techniques 
are  potentially  efficient  because  they  involve  relatively  small  quantities  of  data. 

Contour  models  are  data  compressed  shape  representations,  where  the  model  parameters  arc 
contour  transfonns.  Polygon  models  are  among  the  most  popular.  B-spline  models  are 
powerful  and  versatile  generalizations  of  these  parameters  called  control  points.  Algorithms 
for  computing  arbitrary  shape  moments,  contour  curvature,  shape  projections  and  related 
features  direcdy  from  control  pints,  independent  of  scale,  have  been  developed.  Use  of  these 
algorithms  results  in  an  useful  contour  based  shape  recognition  scheme,  with  classification 
cost  virtually  independent  of  the  number  of  prototypes.  The  scheme  utilizes  complete  shape 
information,  is  scale-translation-rotation  invariant  and  feasibly  realizable.  Error  rates  of  less 
than  1  percent  were  achieved  during  single  font  character  recognition  at  500  dots  per  inch. 
Based  on  similar  principles,  a  method  of  detecting  positions  of  shapes  flaws  was  also 
developed.  See  Publications  [8,  26,  271  ^‘nd  the  PhD  thesis  by  Paglieroni. 

E.  MATHEMATICAL  MORPHOLOGY  TECHNIQUES  FOR  RULE  BASED 
PATTERN  INSPECTION  (JAIN  AND  DARVVISH) 

The  inspection  problem  is  to  find  a  procedure  that  can  detect  whether  an  item  under  test 
conforms  with  predefined  specification  standards  within  an  acceptable  margin  of  tolerance. 
This  research  concerns  shape  inspection  of  nonoverlapping  flat  objects  imaged  against  a  high 
contrast  background. 

Visual  inspection  provides  a  knowledge  rich  environment  where  the  scene  to  be  analyzed  is 
known  a  priori  to  a  high  degree  of  confidence  and  products  are,  usually,  tested  against  clear 
design  criteria.  In  this  research,  the  go.il  is  to  investigate  the  use  of  a  priori  knowledge  about 
the  scene  to  coordinate  and  control  image  segmentation,  interpretation,  and  defect  detection. 
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The  approach  is  composed  of  two  main  steps.  The  first  step  consists  of  proper  segmentation 
and  labeling  of  individual  regions  in  the  image  for  subsequent  interpretation.  Low  level 
morphology  based  techniques  are  used  to  perform  segmentation,  grouping  and  shape 
analysis.  General  as  well  as  specific  knowledge  is  used  to  improve  the  segmentation  and 
interpretation  decisions.  Once  every  region  in  the  image  has  been  identified,  the  second  step 
proceeds  by  testing  different  regions  to  ensure  they  meet  the  design  requirements,  which  arc 
formalized  by  a  set  of  rules.  Morphological  techniques  are  also  used  to  extract  certain 
features  from  the  previously  processed  image  for  rule  verification  purposes. 

The  rule  based  approach  freed  us  from  the  precise  alignment  required  for  the  well  known 
image  subtraction  method.  The  approach  is  also  an  improvement  over  existing  pure 
dimensional  verification  systems  because  it  allows  detection  of  missing  and  erroneous 
features  as  well  as  extraneous  segments  that  might  have  normal  looking  features.  See 
publications  [12,  14,  23]  and  the  PhD  thesis  by  Darwish. 

F.  RADON  TRANSFORM  BASED  IMAGE  PROCESSING  (JAIN,  SANZ, 
BUONOCORE,  HINKLE,  DOHSE,  AZEVEDO) 

This  project  consisted  of  several  subprojects  as  follows: 

1 .  P^E:  A  Parallel  Pipeline  Projection  Engine  (Jain,  Sanz,  Hinkle) 

We  developed  a  novel  architecture  that  makes  real-time  projection-based  image 
processing  a  reality.  The  design  is  founded  on  raster-mode  processing,  which  is 
exploited  in  a  powerful  and  flexible  pipeline.  This  architecture,  dubbed  "P^E" 
(Parallel  Pipeline  Projection  Engine),  supports  a  large  variety  of  image  processing 
and  image  analysis  applications.  In  this  research,  we  were  concerned  with  several 
image  processing  tasks,  such  as:  discrete  approximations  of  the  Radon  and  inverse 
Radon  transform,  among  other  projection  operators;  CT  reconstructions;  2-D 
convolutions;  rotations  and  translations;  etc.  However,  there  is  also  an  extensive  list 
of  key  image  analysis  algorithms  that  are  supported  by  P^E,  thus  making  it  a 
profound  and  versatile  tool  for  projection-based  computer  vision.  Recently,  several 
image  analysis  operators  were  mapped  onto  this  architecture  to  solve  some  important 
automated  inspection  problems.  We  have  yet  to  apply  P^E  to  many  other  unexplored 
image  processing  and  image  analysis  tasks.  Examples  of  these  are:  object 
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recognition,  motion  parameter  computations,  approximation  of  tlie  Fourier  transform 
on  polar  rasters,  etc.  See  publications  [11,  17,  22,  31]. 

2.  Object  Classification  and  Registration  by  Radon  Transform  Based 
Invariants  (Jain,  Sanz,  Yoshida,  Dohse) 

Applications  in  machine  vision  systems  and  digital  object  recognition  often  require 
recognizing  objects  from  a  given  class  regardless  of  position  and  orientation  in 
addition  to  determining  the  angle  of  rotation  and  the  position  of  the  input  object. 

The  magnitude  of  the  Fourier  transform  combined  with  a  circular  harmonics 
expansion  is  used  to  obtain  several  rotation  and  translation  invariant  representations. 
The  Radon  Transform  is  used  as  a  tool  to  calculate  the  Fourier  transform  in  polar 
coordinates.  The  reason  for  going  into  polar  Fourier  space  is  that  the  discrete 
implementation  of  the  integrals  required  for  the  circular  harmonics  expansion  is 
simplified  when  in  polar  space.  In  addition,  at  no  extra  computational  expense,  the 
location  and  rotation  of  an  object  can  be  found. 

The  recognition  scheme  has  been  tested  on  binary  and  gray  level  image  sets. 
Favorable  classification  results  were  obtained  for  both  sets.  See  [20]  and  the  M.S. 
thesis  by  Dohse. 

(;.  MODEL-BASED  COMPUTED  TOMOGRAPHY  FOR  NONDESTRUCTIVE 
EVALUATION  (JAIN,  BUONOCORE,  AZEVEDO) 

This  research  involves  the  use  of  model-based  computer  algorithms  for  solving  the  hollow- 
projections  problem  in  computed  tomography  (CT)  imaging.  In  particular,  we  are  studying 
the  algorithms  best  suited  for  nondestructive  evaluation  (NDE)  of  industrial  objects  where 
applicable  mathematical  models  are  available. 

CT  image  reconstruction  is  a  form  of  multidimensional  signal  nrocessing  that  involves 
extracting  or  reconstructing  a  two-dimensional  (2-D)  planar  density  distribution  through  an 
object  from  measured  data  values  that  are  inexact.  The  measurements  are  the  detected  X-ray 
photons  that  have  been  transmitted  through  a  2-D  p'ane  of  the  object  at  many  different  angles, 
and  the  extracted  information  is  in  the  fomi  of  an  image  of  the  2-D  plane.  Many  standard 
methods  of  image  reconstruction  from  projections  have  been  proposed  and  implemented 
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since  the  development  of  medical  CT  scanners,  but  there  are  still  many  unanswered  questions 
about  the  quality  and  information  content  of  the  reconstructed  images. 

Furthermore,  when  there  are  missing  or  noisy  measurements,  artifacts  can  dominate  the 
resultant  reconstructed  images.  The  hollow-projections  problems  is  a  case  in  point:  some 
rays  cast  through  the  object  are  completely  attenuated  to  zero  by  very  dense  materials.  Our 
interest  is  in  imaging  the  low-density  regions  surrounding  the  highly  attenuating  materials, 
not  the  dense  materials  themselves.  Using  standard  reconstruction  techniques,  however,  the 
low-density  areas  of  interest  will  contain  ghost  anifacts  caused  by  the  missing  data  (zero¬ 
valued  rays).  Industrial  tomography  for  NDE  is  especially  plagued  with  this  problem  since 
the  range  of  densities  in  an  object  can  be  far  larger  than  the  detector  is  capable  of  measuring  . 
On  the  other  hand,  since  the  objects  are  designed  and  built  by  computer,  we  typically  have 
very  detailed  and  accurate  expected  models  of  the  object  profile  from  a  computer-aided  design 
(CAD)  system.  (This  is  in  contrast  to  medical  applications  where  every  body  is  different.) 
Our  premise  is  that  any  additional  information  that  we  can  provide  in  the  form  of 
deterministic  or  stochastic  mathematical  models  should  hep  us  to  improve  the  imaging. 
Though  some  new  techniques  have  been  suggested  with  this  premise  in  mind,  they  have  not 
been  generally  adapted  to  the  hollow-projection  problems  for  NDE. 

This  project  was  recently  started  and  will  continue  after  the  termination  of  this  current  grant. 

H.  AN  EXPANDABLE  VLSI  PROCESSOR  ARRAY  APPROACH  TO  CONTOUR 
TRACING  (HURST,  JAIN,  AGI) 

Contour  Tracing  (or  segmentation)  is  used  in  a  variety  of  image-processing/computer-vision 
applications.  In  certain  applications,  contour  tracing,  implemented  in  software,  takes  too 
long,  even  for  small  images.  Contour  tracing  in  hardware  can  achieve  much  higher  speeds 
especially  if  parallelism  and  other  "speed  up  tricks"  are  used. 

In  the  architecture  proposed,  input  image  needs  to  be  thresholded  and  converted  into  binary 
pixels.  Then  the  image  is  broken  up  into  2M  by  M  blocks,  where  each  processor  loads  a 
single  block.  The  contour  tracer  consists  of  a  set  of  two  processor  arrays  which  work 
independently.  As  one  array  of  processors  traces  out  the  contours  contained  in  its  portion  of 
the  image  and  broadcasts  the  contour  information  to  the  post-processor,  the  second  set 
extracts  the  edges  (or  cracks)  that  are  to  be  traced.  These  two  arrays  switch  operations  once 
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the  loading  of  the  on-board  RAM  is  complete.  A  post-processor  links  all  the  partial  contours 
into  a  set  of  continuous  contours  which  can  be  used  for  further  processing. 

The  compression  ratios  obtained  by  the  processor  array  were  no  worse  and  sometimes  better 
than  the  maximum  possible  by  Run  Length  Coding  (RLC)  of  the  CCITT  documents.  The 
compression  ratio  can  be  further  increased  by  the  linking  of  the  partial  contours  and  run- 
length  encoding  the  direction  vectors.  The  tracing  time  was  reduced  significantly  by  keeping 
track  of  the  first  edge  encountered  on  each  line.  See  [24]  and  the  M.S.  thesis  by  Agi. 

I.  A  RASTER  SCAN  ARCHITECTURE  FOR  BOUNDARY-BASED 
SEGMENTATION  (JAIN,  SANZ,  APFFEL) 

A  novel  hardware  architecture  for  extracting  region  boundaries  in  two  raster  scan  passes 
through  a  binary  image  has  been  developed.  The  first  pass  is  a  statistic  gathering  phase 
wherein  data  is  collected  regarding  the  size  of  each  object  contour.  This  information  is  used 
by  the  hardware  to  dynamically  allocate  available  memory  for  storage  of  boundary  codes.  In 
the  second  raster  pass,  the  same  architecture  constructs  lists  of  Grid-Joint  codes  to  represent 
the  perimeter  pixels  of  each  object.  These  codes,  referred  to  variously  as  "crack"  codes  or 
"raster-chain"  codes,  are  later  decoded  by  the  hardware  to  reproduce  the  ordered  sequence  of 
coordinates  surrounding  each  object.  This  list  of  coordinates  is  useful  for  a  variety  of  shape 
recognition  and  manipulation  algorithms  which  utilize  boundary  information. 

The  proposed  VLSI  architecture  has  been  simulated  in  software.  Measurements  on  the 
coding  efficiency  of  the  basic  algorithm  as  well  as  estimates  of  the  overall  chip  complexity 
have  been  explored.  See  [21]  and  the  M.S.  thesis  by  Apffel. 

J.  BLOCK  PROCESSING  OF  IMAGES  WITH  SIMD  MULTIPROCESSORS 
(JAIN,  SRINIVASAN,  VAN  TIGHEM,  CHIN) 

Programmable  Single  Instruction  Multiple  Data  (SIMD)  stream  chips  can  meet  the  high 
throughput  requirements  of  many  image  processing  applications.  Because  of  the  limited 
memory  capacity  of  these  devices,  processing  image  data  in  blocks  becomes  a  very  practical 
and  efficient  way  to  utilize  the  full  capacity  of  these  chips  and  achieve  real-time  image 
processing  at  a  reasonable  cost.  Specifically,  the  NCR-Geometric  Arithmetic  Parallel 
Processor  (GAPP)  was  investigated  and  tested  with  various  well-known  enhancement 
algorithms.  See  [15]  and  the  M.S.  thesis  by  Van  Tighem. 
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K.  AN  UNIFIED  DCT/IDCT  ARCHITECTURE  FOR  VLSI  IMPLEMENTATION 
(JAIN,  CURRENT,  SRINIVASAN,  PARKHURST,  FARRELLE,  GULIANI, 
GRISHAW) 

This  project  involved  the  design,  layout,  and  simulation  of  a  VLSI  chip.  This  chip  performs 
the  discrete  cosine  transform  (DCT)  and  inverse  discrete  cosine  transform  (IDCT).  Two, 
three,  and  four-  input  nor  gates,  a  register,  a  three-bit  counter,  a  15-bit  adder  as  well  as  a 
fixed  coefficient  multiplier  were  designed  and  layed  out.  These  layouts  as  well  as  the  total 
chip  layout  were  done  using  MAGIG,  a  CAD  tool  on  the  SUN  workstations.  Simulations, 
using  SPICE  and  ESIM,  were  performed  on  component  parts  of  the  chip  as  well  as  the  chip 
itself  The  design  and  layout  of  the  chip  is  finished  while  simulations  on  the  chip  are  almost 
complete.  See  [13,  24]  and  the  M.S.  theses  by  Parkhurst  andGrishaw. 

L.  SKELETONIZATION  AND  VECTOR  CONVERSION  OF  DOCUMENT 
IMAGES  (JAIN  AND  BRANDT) 

This  research  examines  the  midline  representation  of  an  object.  The  midline,  or  medical  axis, 
can  be  defined  as  the  locus  of  points  in  an  object  which  are  equidistant  to  at  least  two 
boundary  points.  The  midline  is  a  contour  that  falls  along  the  central  axis  of  the  shape. 
Using  this  contour,  together  with  the  object's  thickness  at  each  point  along  the  contour,  it  is 
possible  to  reconstruct  the  object.  It  is  possible  to  encode  the  midline  into  a  vector 
representation  so  as  to  achieve  efficient  storage  and  reconstruction. 

The  following  questions  are  being  addressed: 

A.  How  much  compression  can  be  achieved  while  retaining  accurate  reconstruction? 

B.  How  are  geometric  transformations  applied  to  the  image  using  the  vector 
representation? 

C.  What  are  the  computational  requirements  for  conversion  and  manipulation  of  this 
representation? 

This  is  a  continuing  project.  See  the  M.S.  thesis  by  Brandt. 


